Lower bounds on minimax rates for nonparametric regression with additive sparsity and smoothness

We study minimax rates for estimating high-dimensional nonparametric regression models with sparse additive structure and smoothness constraints. More precisely, our goal is to estimate a function f∗ : R → R that has an additive decomposition of the form f(X1, . . . ,Xp) = ∑ j∈S h ∗ j (Xj), where each component function h ∗ j lies in some class H of “smooth” functions, and S ⊂ {1, . . . , p} is an unknown subset with cardinality s = |S|. Given n i.i.d. observations of f∗(X) corrupted with additive white Gaussian noise where the covariate vectors (X1,X2,X3, ...,Xp) are drawn with i.i.d. components from some distribution P, we determine lower bounds on the minimax rate for estimating the regression function with respect to squared-L(P) error. Our main result is a lower bound on the minimax rate that scales as max ( s log(p/s) n , s ǫ 2 n(H) ) . The first term reflects the sample size required for performing subset selection, and is independent of the function class H. The second term s ǫn(H) is an s-dimensional estimation term corresponding to the sample size required for estimating a sum of s univariate functions, each chosen from the function class H. It depends linearly on the sparsity index s but is independent of the global dimension p. As a special case, if H corresponds to functions that are m-times differentiable (an m-order Sobolev space), then the s-dimensional estimation term takes the form sǫn(H) ≍ s n−2m/(2m+1). Either of the two terms may be dominant in different regimes, depending on the relation between the sparsity and smoothness of the additive decomposition.